Enforce no real filesystem writes during unit tests #700
+39
−83
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…geProvider::new_overlay()
arrayka
changed the title
arrayka
commented
Jan 23, 2026
diskann-disk/src/utils/aligned_file_reader/storage_provider_aligned_file_reader.rs
Show resolved
Hide resolved
diskann-disk/src/utils/aligned_file_reader/storage_provider_aligned_file_reader.rs
Outdated
Show resolved
Hide resolved
diskann-disk/src/utils/aligned_file_reader/storage_provider_aligned_file_reader.rs
Show resolved
Hide resolved
7 tasks
Contributor
* Initial plan * Address PR feedback: revert comments and refactor test helper functions Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com> * Revert test_index_path() to return String and remove .to_string_lossy() Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com> * Remove test_index_os_path() and use test_index_path() instead Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com> * Revert commit 90934ad Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com>
| pub fn benchmark_copy_aligned_data(c: &mut Criterion) { | ||
| let tmp_dir = TempDir::with_prefix(BENCHMARK_ID).expect("Failed to create temporary directory"); | ||
| // Use physical file system rather than memory for testing the actual disk read/write | ||
| #[allow(clippy::disallowed_methods)] |
Contributor
There was a problem hiding this comment.
You can use
#[expect(clippy::disallowed_methods, "reason")]to do two things:
expectwill complain if disallowed method is not actually used (helping with maintanence)- The following reason helps document why an exception is made.
| pub fn benchmark_copy_aligned_data_iai() { | ||
| let tmp_dir = TempDir::with_prefix(BENCHMARK_ID).expect("Failed to create temporary directory"); | ||
| // Use physical file system rather than memory for testing the actual disk read/write | ||
| #[allow(clippy::disallowed_methods)] |
Contributor
There was a problem hiding this comment.
Same comment here with expect instead of allow.
| let storage_provider = VirtualStorageProvider::new(vfs); | ||
| let pq_pivots_path: &str = "test_data/sift/siftsmall_learn_pq_pivots.bin"; | ||
| let storage_provider = VirtualStorageProvider::new_overlay(workspace_root); | ||
| let pq_pivots_path: &str = "/test_data/sift/siftsmall_learn_pq_pivots.bin"; |
Contributor
There was a problem hiding this comment.
Something we should also do (maybe in another PR) is to replace all uses of test_data with the file system resolution in diskann-utils. I think a few fell through the cracks when test data was consolidated.
Contributor
Author
There was a problem hiding this comment.
It makes sense. I've created an issue to track this: #704
| impl VirtualStorageProvider<OverlayFS> { | ||
| /// Create a two-layer overlay filesystem with an in-memory filesystem for writes | ||
| /// on top of the physical filesystem for reads. | ||
| pub fn new_overlay<P: AsRef<std::path::Path>>(path: P) -> Self { |
Contributor
There was a problem hiding this comment.
Do you see any value in making VirtualStorageProvider::new much harder to call to encourage use of new_overlay or new_memory? Maybe with like a new_physical and then linting that with clippy?
Contributor
Author
There was a problem hiding this comment.
Yes.
I've created an issue to track this: #703
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
What
vfs::PhysicalFS::newto the list of disallowed methods inclippy.toml